Combining stereo image layers for display

ABSTRACT

A system and method for stereoscopic pair layers includes aligning a left eye image and a right eye image of a first stereo image pair layer according to a first calibrated offset to produce an aligned first stereo image pair layer that appears at a first depth in a display environment. A left eye image and a right eye image of a second stereo image pair layer are aligned according to a second calibrated offset to produce an aligned second stereo image pair layer that appears at a second depth in the display environment that is different than the first depth. The aligned first stereo image pair layer and the aligned second stereo image pair layer combined to produce a calibrated stereoscopic image that is suitable for display.

BACKGROUND

1. Field of the Invention

The present invention relates generally to the display of stereo imagesand more specifically to a system and method for combining stereo imagelayers during playback.

2. Description of the Related Art

Historically, stereo images were displayed using two projectors, oneprojecting images viewed only by the left eye, and the other projectingimages viewed only by the right eye. The left and right eye images werethen manually aligned to control the convergence point of the stereoimage produced by each left-eye/right-eye image pair projected onto ascreen. More recently, the use of single lens stereo projection systems,has made adjusting the convergence point of a stereo image impossiblewhen the stereo image is displayed.

When stereo image content is created, a displayed image size is assumed,and the perceived distance from the viewer where objects appear in ascene (e.g., in front of the screen, coincident with the screen, orbehind the screen) is controlled by the author of the stereo imagecontent assuming a predetermined alignment between the left and righteye image pairs. The predetermined alignment corresponds to the assumeddisplayed image size. One problem with such as approach is that, whenthe image content is displayed on a smaller or larger screen, theperceived distance of the objects in the scene is affected as a resultof an increased or decreased image pair alignment. In some cases, theincreased or decreased alignment causes a viewer to experiencediscomfort due to eye strain or eye fatigue. This problem is furtherillustrated below in FIGS. 1A-1F.

FIG. 1A is an illustration of a prior art stereoscopic geometry with aconvergence depth 111 that equals the display surface depth. A left eye101 and right eye 102 are separated by an interoccular separation 105.An object that has no separation for the left eye image relative to theright eye image is perceived to appear coincident with display surface104. In other words, objects that are not separated in the left andright eye images appear in the plane of the display surface. Theintersection of the view vectors from left eye 101 and right eye 105 isconvergence point 110, which is shown as coincident with display surface104. During construction of a stereoscopic image, the alignment of leftand right eye views for objects in the scene is determined by the authorand controls the perceived depth at which the objects appear to aviewer.

FIG. 1B is an illustration of a prior art stereoscopic geometry with aconvergence depth 122 that is less than the depth of display surface104. An object that is offset to the left by stereo image offset 125 inthe right eye image compared with the left eye image appears to theviewer at convergence point 120 (i.e., in front of display surface 104).When the right eye image and left eye image are scaled up, theconvergence point shifts closer to the viewer. FIG. 1C is anillustration of a prior art stereoscopic geometry for a larger displaysurface 114 with a convergence depth 123 that is less than the intendedconvergence depth 122. As shown, as stereo images are scaled to fit thelarger display surface 114 (relative to the display surface 104), stereoimage offset 125 is scaled to equal larger stereo image offset 124. Theviewer may experience discomfort due to increased cross-eyedness as aresult of larger stereo image offset 124 since objects in the sceneappear at convergence point 121 instead of at convergence point 120.

FIG. 1D is an illustration of a prior art stereoscopic geometry with aconvergence depth 127 that is greater than the depth of display surface104. An object that is offset to the right by stereo image offset 126 inthe right eye image compared with the left eye image appears to theviewer at convergence point 123 (i.e., behind display surface 104). Whenthe right eye image and left eye image are scaled up, the convergencepoint shifts further from the viewer. Objects that are separated byinteroccular separation 105 in the left eye and right eye images appearat infinity. FIG. 1E is an illustration of a prior art stereoscopicgeometry for larger display surface 114 with a convergence depth 128that is greater than the intended convergence depth 127 since theseparation between the left eye and right eye images is greater thaninteroccular separation 105. As shown, as stereo images are scaled tofit the larger display surface 114 (relative to the display surface 104)stereo image offset 126 is scaled to equal larger stereo image offset129 as the stereo images are scaled to fit larger display surface 114.The viewer more than likely may see double images or experiencediscomfort due to increased divergence as a result of larger stereoimage offset 129 since objects in the scene appear beyond infinityinstead of at convergence point 123.

Similarly, when the right eye image and left eye image are scaled down,the convergence point shifts closer to the viewer and objects that wereintended to appear behind the screen may appear coincident with thescreen. FIG. 1F is an illustration of prior art stereoscopic geometryfor a smaller display surface 134 with convergence depth 137 that isless than the intended convergence depth 127. As shown, as stereo imagesare scaled to fit the smaller display surface 134 (relative to thedisplay surface 104) stereo image offset 126 is scaled to equal smallerstereo image offset 136. Again, objects in the scene do not appear atthe depths that were intended when the stereoscopic content was createdsince the stereo image offset used to produce the content does not matchsmaller stereo image offset 136.

As the foregoing illustrates, what is needed in the art is the abilityto modify the alignment between stereoscopic images based on the size ofthe display surface when a single lens stereo projection system is used.

SUMMARY

A system and method for combining stereoscopic image layers for displayallows for calibration of the stereo image content for each displayenvironment. An offset between displayed stereo image pairs may becharacterized and adjusted for each stereo image pair layer to controlthe perceived depth at which each layer appears to a viewer. The stereoimage pair layers are combined to produce a combined stereo image pairfor display. The stereo image content is typically authored assuming aparticular display image size. When the stereo image content isdisplayed in a display environment that does not conform to theparticular display image size the offset between displayed stereo imagepair layers is increased or decreased, resulting in a viewing experiencethat is different than intended. In some cases, the viewer mayexperience discomfort due to eye strain or eye fatigue. Adjusting theoffset during the playback of the stereo image content may improve theviewer experience.

A system and method for combining stereoscopic pair layers includesaligning a left eye image and a right eye image of a first stereo imagepair layer according to a first calibrated offset to produce an alignedfirst stereo image pair layer that appears at a first depth in a displayenvironment, aligning a left eye image and a right eye image of a secondstereo image pair layer according to a second calibrated offset toproduce an aligned second stereo image pair layer that appears at asecond depth in the display environment that is different than the firstdepth, and combining the aligned first stereo image pair layer and thealigned second stereo image pair layer to produce a calibratedstereoscopic image that is suitable for display.

Various embodiments of the invention include a stereoscopic image systemfor combining stereoscopic pair layers. The system includes a displayprocessor configured to align a left eye image and a right eye image ofa first stereo image pair layer according to a first calibrated offsetto produce an aligned first stereo image pair layer that appears at afirst depth and align a left eye image and a right eye image of a secondstereo image pair layer according to a second calibrated offset toproduce an aligned second stereo image pair layer that appears at asecond depth that is different than the first depth. The displayprocessor is configured to combine the aligned first stereo image pairlayer and the aligned second stereo image pair layer to produce acalibrated stereoscopic image that is suitable for display.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1A is an illustration of a prior art stereoscopic geometry with aconvergence depth that equals the display surface depth;

FIG. 1B is an illustration of a prior art stereoscopic geometry with aconvergence depth that is less than the display surface depth;

FIG. 1C is an illustration of a prior art stereoscopic geometry for alarger display surface with a convergence depth that is less than thedisplay surface depth;

FIG. 1D is an illustration of a prior art stereoscopic geometry with aconvergence depth that is greater than the display surface depth;

FIG. 1E is an illustration of a prior art stereoscopic geometry for alarger display surface with convergence depth that is greater than thedisplay surface depth;

FIG. 1F is an illustration of a prior art stereoscopic geometry for asmaller display surface with convergence depth that is less than thedisplay surface depth;

FIG. 2A is an illustration of stereoscopic geometry for a larger displaysurface using an adjusted stereo image offset, according to oneembodiment of the present invention;

FIG. 2B is an illustration of stereoscopic geometry for a smallerdisplay surface using an adjusted stereo image offset, according to oneembodiment of the present invention;

FIG. 2C is an illustration of a display system with associatedstereoscopic geometry, according to one embodiment of the presentinvention;

FIG. 3A is a flow diagram of method steps for characterizing andadjusting a convergence point for playback of a stereo content,according to one embodiment of the present invention;

FIG. 3B is a flow diagram of method steps for characterizing andadjusting a convergence point for playback of a stereo content using adynamic adjustment factor, according to another embodiment of thepresent invention;

FIG. 4A is an illustration of stereoscopic geometry for a display systemconfigured to combine multiple stereoscopic pair layers, according toone embodiment of the present invention;

FIG. 4B is a flow diagram of method steps for combining multiplestereoscopic pair layers for playback of a stereo content, according toone embodiment of the present invention;

FIG. 4C is a flow diagram of method steps for characterizing, aligning,and combining multiple stereoscopic pair layers for playback of a stereocontent, according to one embodiment of the present invention; and

FIG. 5 is a block diagram of a system configured to implement one ormore aspects of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the invention provide a method and system for adjustingthe convergence point of a stereoscopic image displayed on a displayenvironment by determining a characterized offset between a left eyeimage and a right eye image of a stereo image pair in a displayenvironment. When the characterized offset does not equal apredetermined offset value an adjustment factor is determined. Theadjustment factor is applied to the characterized offset to adjust theconvergence point for the display environment by producing a calibratedoffset between the left eye image and the right eye image of the stereoimage pair.

Additionally, embodiments of the invention provide a method and systemfor combining stereoscopic pair layers, as described in conjunction withFIGS. 4A, 4B, and 4C. A left eye image and a right eye image of a firststereo image pair layer is aligned according to a first calibratedoffset to produce an aligned first stereo image pair layer that appearsat a first depth in a display environment. A left eye image and a righteye image of a second stereo image pair layer are aligned according to asecond calibrated offset to produce an aligned second stereo image pairlayer that appears at a second depth in the display environment that isdifferent than the first depth. The aligned first stereo image pairlayer and the aligned second stereo image pair layer combined to producea calibrated stereoscopic image that is suitable for display.

FIG. 2A is an illustration of stereoscopic geometry for a larger displaysurface 214 using an adjusted stereo image offset 226, according to oneembodiment of the present invention. A left eye 201 and a right eye 202are separated by an interoccular separation 205. Rather than simplyscaling the left eye image and the right eye image for display on largerdisplay surface 214, thereby increasing the stereo image offset, thealignment of the left eye image and right eye image is modified toreduce the actual stereo image offset to equal calibrated stereo imageoffset 226. Calibrated stereo image offset 226 equals a predeterminedoffset that corresponds to the convergence point 223 and convergencedepth 227 originally intended by the content author. With thistechnique, objects in the stereoscopic image intended to appear at aparticular depth, based on an assumed screen size used to create theimage content, will appear at that depth when the image is scaled to fitlarger display surface 214. Consequently, the likelihood that a viewerwill experience eyestrain resulting from scaling up stereo images isreduced.

Advantageously, since calibrated stereo image offset 226 is used toalign stereo image pairs as they are scaled up or down at the time ofdisplay, stereo image pairs authored for particular screen sizes may beadjusted using different image offsets and displayed on a variety ofdifferent screen sizes. Thus, different sets of image pairs do not needto be created for each target screen size.

FIG. 2B is an illustration of stereoscopic geometry for a smallerdisplay surface 234 using a calibrated stereo image offset 236,according to one embodiment of the present invention. Rather than simplyscaling the left eye image and the right eye image for display onsmaller display surface 234, thereby decreasing the stereo image offset,the alignment of the left eye image and right eye image is modified toincrease the actual stereo image offset to equal calibrated stereo imageoffset 226. Again, calibrated stereo image offset 226 equals apredetermined offset that corresponds to the convergence point 233 andconvergence depth 237 originally intended by the content author. Similarto that described above, with this technique objects in the stereoscopicimage intended by the content author to appear at a particular depth,based on an assumed screen size used to create the image content, willappear at that particular depth when the content is scaled to fillsmaller display surface 234. In this fashion, the likelihood that aviewer will experience eyestrain resulting from scaling down stereoimages is reduced.

FIG. 2C is an illustration of a display system 250 with associatedstereoscopic geometry, according to one embodiment of the presentinvention. Display system 250 may include a media center and projectorthat is configured to project stereo images onto a display surface 200.In an alternative embodiment, display surface 200 may be a flat paneldisplay, and display system 250 may provide data to produce an image ondisplay surface 200. In the configuration shown, a viewer may bepositioned either in front of or behind display surface 200.Characterized offset 252 is the actual image offset produced by anyscaling of the stereo images being displayed on display surface 200. Thevalue of characterized offset 252 is first determined, and then the lefteye image and the right eye image are aligned by modifying the (actualimage) offset to equal a calibrated offset 254. Calibrated offset 254should equal the predetermined offset that was used to create the stereoimage content. As persons skilled in the art will appreciate, when theoffset requires no adjustment to equal calibrated offset 254, displaysurface 200 is the display size used to create the stereo image content.

As also shown in FIG. 2C, characterized offset 252 is larger thancalibrated offset 254, indicating that the stereo image pair was scaledup by display system 250 for display on display surface 200. Displaysystem 250 is configured to adjust the alignment between the left andright eye images by modifying the offset that initially equalscharacterized offset 252, to match calibrated offset 254 beforeprojecting the stereo image pair onto display surface 200. Typically,the alignment is adjusted in the horizontal direction.

FIG. 3A is a flow diagram of method steps for characterizing andadjusting a convergence point for playback of a stereo content,according to one embodiment of the present invention. Although themethod steps are described with respect to FIGS. 2A-2C, persons skilledin the art will understand that any system configured to perform themethod steps, in any order, falls within the scope of the presentinvention.

The method begins in step 300, where characterization data for a displayenvironment is determined. Several techniques may be used to determinethe characterization data. For example, the stereo image offset betweena left and a right eye image of a stereo pair, such as characterizedoffset 252, may be measured to produce the characterization data. Thecharacterization data may be determined by projecting a stereoscopictest pattern onto display surface 200 and physically measuring acharacteristic of the displayed stereoscopic text pattern to determinecharacterized offset 252. The measured characteristic can be a distancebetween elements of the stereoscopic test pattern, such as crosshairs,circles or other shapes, a diameter or radius of a shape, and the like.Alternatively, characterized offset 252 is encoded as part of thestereoscopic test pattern, such that the displayed stereoscopic testpattern visually indicates the value of characterized offset 252.Various different values of characterized offset 252 are encoded in thestereoscopic test pattern, and the value that corresponds to theparticular display environment is visible in the displayed stereoscopictest pattern.

Alternatively, in step 300, the characterization data may be computedbased on the screen size, the predetermined offset used to create thestereoscopic image content, and the size of display surface 200. Thepredetermined offset may be a standard human interoccular distance, suchas interoccular distance 205. In other embodiments of the presentinvention, the characterization data may be determined using a physicaldevice to calibrate display system 250. The physical device may indicatethe length of calibrated offset 254 and the offset between the left andright eye images may be dynamically adjusted by display system 250 whilea stereoscopic test pattern is displayed. When the offset matchescalibrated offset 254, the characterization data can be determined bydisplay system 250 based on the adjustment that was applied.

If, in step 305, a determination is made that the characterized offset252 matches the calibrated offset 254, i.e., that the actual offset iscorrect, then no adjustment of the alignment between the left and righteye images is needed to produce the convergence point intended by thecontent author. The method then proceeds directly to step 330, where thestereoscopic images are displayed using characterized offset 252.Otherwise, if characterized offset 252 is determined to not matchcalibrated offset 254, then in step 310, display system 250 determinesif calibration is enabled for displaying the stereoscopic content. Ifcalibration is not enabled, then the method proceeds directly to step330, where the stereoscopic images are displayed using characterizedoffset 252. In some use scenarios, displaying the stereoscopic contentwithout adjusting the offset between stereo pairs may be desirable. Forexample, characterized offset 252 may be considered to be correct whenit is within a threshold of calibrated offset 254. The threshold may bea percentage of calibrated offset 254 or a value that is fixed orspecified by a viewer, content author, or display system 254.

If, in step 310, display system 250 determines that calibration isenabled, then in step 320, an adjustment factor is determined. Theadjustment factor may be computed using the characterization data. Forexample, the adjustment factor may be a ratio of the size of displaysurface 200 to the screen size assumed by the content author whencreating the image content or a ratio of characterized offset 252 tocalibrated offset 254. The adjustment factor may also be determined whenthe physical device is used to calibrate display system 250.

In step 325, the adjustment factor is applied by display system 250 toproduce an adjusted offset between the left and right eye images of eachstereo pair in the stereoscopic content. Specifically, using theadjustment factor, display system 250 adjusts the offset, as needed, toequal calibrated offset 254. In step 330, the images are output bydisplay system 250 for display on display surface 200 using calibratedoffset 254. Just as a threshold may be used to determine whether or notthe offset is correct, a threshold may be used to adjust the offset tobe within a certain range of calibrated offset 254. In other words,characterized offset 252 may be adjusted to equal calibrated offset 254plus or minus the threshold. The threshold may be a percentage ofcalibrated offset 254 or a value that is fixed or specified by a viewer,content author, or display system 250.

In some cases, all of the stereo image pairs of the stereo content maybe displayed using a single adjustment factor. In other cases, it may bedesirable to dynamically change the adjustment factor. For such apurpose, additional characterization data may be encoded within thestereo content. For example, an author of the stereo content may want tochange a viewer's focus from an actor's face, having an associatedconvergence point on display surface 200, to distant vista, having noconvergence point, with separation equal to the interoccular separation.The author may change the convergence point for each frame (i.e., eachstereo image pair), each shot, or at any other desired frequency. Theadditional characterization data that is encoded within the stereocontent represents changes in the convergence point, i.e., animates theconvergence point.

FIG. 3B is a flow diagram of method steps for characterizing andadjusting a convergence point for playback of a stereo content using adynamic adjustment factor, according to another embodiment of thepresent invention. Although the method steps are described in referenceto FIGS. 2A-2C, persons skilled in the art will understand that anysystem configured to perform the method steps, in any order, fallswithin the scope of the present invention.

In one embodiment, steps 300, 305, 310, and 320 are first performed asdescribed above in FIG. 3A. Therefore, no additional description isprovided herein. The method then process to step 340, where displaysystem 250 determines if the stereo content includes additionalcharacterization data indicating that the adjustment factor used tocontrol the convergence point should be determined dynamically. If not,the method proceeds directly to step 350. Otherwise, in step 345, theadjustment factor is updated to reflect the convergence point specifiedby the additional characterization data. In step 350, the adjustmentfactor is applied by display system 250 to produce calibrated offset 254between the left and right eye images of each stereo pair in the stereocontent. When the adjustment factor is dynamic, reflecting updates fromstep 345, calibrated offset 254 varies as the adjustment factor varies,thereby producing a dynamic convergence point. In step 355, the stereoimage pair is output by display system 250 for display on displaysurface 200 using calibrated offset 254. Steps 340, 345, 350, and 355are repeated for each stereo pair in the stereo content.

Adjusting the offset between the stereo images enables display system250 to replicate the viewer experience that was intended by the authorof the stereo content. Additionally, viewer discomfort due to eye strainor eye fatigue may be reduced or avoided when the stereo content isscaled up or down from the intended display size. Finally, performingthe offset adjustment in display system 250 allows for stereo content tobe retargeted to different display sizes without regenerating the stereocontent.

Combining Stereoscopic Image Pair Layers

During the generation of stereo image content an object in the left eyeand right eye images are offset based on how close or far from a viewerthe object should appear when the stereo image content is displayed.Once the objects are composited to generate each left eye and right eyeimage pair the relative depths of different objects is fixed. In otherwords an object in the background will appear behind an object in theforeground of a scene. After the objects are composited, the object inthe background cannot be made to appear in front of an object in theforeground.

In order to provide greater control over the relative depths of objectsin the stereo image content, two or more stereoscopic image layers maybe generated that are combined at the time of display. For example, afirst layer may include a stereo image pair with text for the title andcredits and a second layer may include a stereo image pair with abackground. An offset between the left eye image and right eye image ofeach stereo pair layer can be adjusted to control the depth at whicheach layer appears to a viewer. A first offset for the first layer maybe adjusted so that the first layer appears in front of, coincidentwith, or behind the second layer. After the offsets for each layer areadjusted, the layers are combined to produce a stereo image pair fordisplay.

In addition to providing the ability to control the relative depth ofthe different layers, the offsets can be adjusted to reduce viewerdiscomfort due to eye strain or eye fatigue when the stereo imagecontent is displayed in a display environment that does not conform tothe display image size used to author the content. Adjusting the offsetduring the playback of the stereo image content to compensate forscaling the stereo content for a particular display environment andallow control of the convergence point position to match the positionintended by the content author.

FIG. 4A is an illustration of stereoscopic geometry for a display systemconfigured to combine multiple stereoscopic pair layers, according toone embodiment of the present invention. Three stereo pair layers areshown in FIG. 4A, a screen stereo pair layer 402 that is coincident witha display surface 414, a behind stereo pair layer 401 and a front stereopair layer 403. Screen stereo pair layer 402, behind stereo pair layer401, and front stereo pair layer 403 each include a left eye image and aright eye image. When each layer is viewed using a calibrated offsetthat matches an intended offset for that layer, the layers are perceivedby a viewer to appear at display surface 414, behind convergence depth421, and front convergence depth 423, respectively.

Front stereo pair layer 403, screen stereo pair layer 402, and behindstereo pair layer 401 are combined to produce a combined left eye stereoimage and a combined right eye stereo image. The stereo pair layers maybe combined in a front-to-back order or a back-to-front order to producethe combined stereo image pair. In some embodiments, only two stereopair layers are combined to produce the stereo image pair for display.In other embodiments, more than two or three stereo image pair layersare combined to produce the stereo image pair for display.

FIG. 4B is a flow diagram of method steps for combining multiplestereoscopic pair layers for playback of a stereo content, according toone embodiment of the present invention. The steps shown in FIG. 4B areperformed to produce a combined stereo image pair that is suitable fordisplay in a stereoscopic display environment, such as display system250. Although the method steps are described in reference to FIG. 4A,persons skilled in the art will understand that any system configured toperform the method steps, in any order, falls within the scope of thepresent invention.

In step 400 a left eye image and a right eye image of a first stereopair layer, e.g., front stereo pair layer 403, screen stereo pair layer402, behind stereo pair layer 401, and the like, is aligned according toa calibrated offset that corresponds to front convergence depth 423. Instep 405 a left eye image and a right eye image of a first stereo pairlayer, e.g., front stereo pair layer 403, screen stereo pair layer 402,or behind stereo pair layer 401, is aligned according to a calibratedoffset that corresponds to an respective convergence depth, e.g., frontconvergence depth 423, screen convergence depth 422, or behindconvergence depth 421. In step 410 the aligned first stereo image pairlayer is combined with the aligned other stereo image pair layer toproduce a calibrated stereoscopic layer that is suitable for display.

In step 415 the method determines if another layer is provided toproduce the calibrated stereoscopic image, and, if so, steps 405, 410,and 415 are repeated for each additional stereo pair layer to producethe combined stereo pair image. When all of the stereo pair layers havebeen combined, in step 420 the combined stereo image pair is displayedin the display environment. Note, that in some embodiments, all of thestereo pairs may be aligned and then all combined to produce thecombined stereo pair image pair.

As previously described, when the stereo pair images are scaled fordisplay on a display surface that is larger or smaller than the intendeddisplay size, the actual offset between the left eye image and the righteye image within a stereo pair may not equal the calibrated offset ofthe display environment. Similarly, when stereo pair layers are scaledfor display, the actual offset for one or more of the stereo pair layersmay not equal the respective calibrated offset for the displayenvironment. Consequently, the convergence depths for each layer or thecombined stereo image may cause viewer eye strain or eye fatigue whenused to display the stereo image rather than the calibrated offsets.

FIG. 4C is a flow diagram of method steps for characterizing, aligning,and combining multiple stereoscopic pair layers for playback of a stereocontent, according to one embodiment of the present invention. The stepsshown in FIG. 4C are performed to produce a combined stereo image pairthat is suitable for display in a stereoscopic display environment, suchas display system 250. Although the method steps are described inreference to FIG. 4A, persons skilled in the art will understand thatany system configured to perform the method steps, in any order, fallswithin the scope of the present invention.

The method begins in step 425, where characterization data for a displayenvironment is determined. As previously described with reference tostep 300 of FIG. 3A, several techniques may be used to determine thecharacterization data. In step 430, an adjustment factor is determined.The adjustment factor may be computed using the characterization data.For example, the adjustment factor may be a ratio of the size of displaysurface 414 to the screen size assumed by the content author whencreating the image content or a ratio of the characterized offset to thecalibrated offset for the stereo pair layer. The adjustment factor mayalso be determined by increasing or decreasing the offset that initiallyequals the characterized offset to equal the calibrated offset andmeasuring the amount that the offset was increased or decreased.

In step 435 the method determines if another layer is provided toproduce the calibrated stereoscopic image, and, if so, steps 425, 430,and 435 are repeated for each additional stereo pair layer to determinethe characterization data and adjustment factor for each stereo pairlayer. When the characterization data and adjustment factor for eachstereo pair layer has been determined, in step 440, the adjustmentfactor for the first stereo pair layer is applied by display system 250to produce an aligned first stereo image pair layer. Specifically, usingthe adjustment factor, display system 250 adjusts the offset for thefirst stereo pair layer, as needed, to equal the calibrated offset forthe first stereo pair layer. In step 445, the adjustment factor foranother stereo pair layer is applied by display system 250 to produce analigned other stereo image pair layer. Specifically, using theadjustment factor, display system 250 adjusts the offset for the otherstereo pair layer, as needed, to equal the calibrated offset for theother stereo pair layer.

A threshold may be used to adjust the offset for each stereo pair layerto be within a certain range of the calibrated offset for that stereopair layer. In other words, the offset may be adjusted to equal thecalibrated offset plus or minus the threshold. The threshold may be apercentage of the calibrated offset or a value that is fixed orspecified by a viewer, content author, or display system 250.

In step 450 the aligned first stereo image pair layer is combined withthe aligned other stereo image pair layer to produce a calibratedstereoscopic layer that is suitable for display. If, in step 455 themethod determines that another layer is provided to produce thecalibrated stereoscopic image then, steps 445, 450, and 455 are repeatedfor each additional stereo pair layer to produce the combined stereopair image. When all of the stereo pair layers have been combined, instep 460 the combined stereo image pair is displayed in the displayenvironment. Note, that in some embodiments, all of the stereo pairs maybe aligned and then all combined to produce the combined stereo pairimage pair.

Multiple stereo pair layers may be combined at the time of display tocontrol the depth at which objects in the scene are perceived to appearto a viewer. The offset between a left eye image and a right eye imagemay be adjusted for each layer to control the convergence point for eachlayer. The offset for each stereo pair layer may be characterized in thedisplay environment and adjusted to improve the viewer experience. Whenthe stereo pair layers are scaled for display on a surface that islarger or smaller than the size of the display that was assumed duringgeneration of the stereo pair layers, the offset for each layer may becalibrated to reduce viewer discomfort due to eye strain or eye fatigue.

FIG. 5 is a block diagram of a system 500 configured to implement one ormore aspects of the present invention. System 500 may be a personalcomputer, video game console, personal digital assistant, or any otherdevice suitable for practicing one or more embodiments of the presentinvention. Any number of the elements shown in system 500 may beincluded in display system 250.

As shown, system 500 includes a central processing unit (CPU) 502 and asystem memory 504 communicating via a bus path that may include a memorybridge 505. CPU 502 includes one or more processing cores, and, inoperation, CPU 502 is the master processor of system 500, controllingand coordinating operations of other system components. System memory504 stores software applications and data for use by CPU 502. CPU 502runs software applications and optionally an operating system. Memorybridge 505, which may be, e.g., a Northbridge chip, is connected via abus or other communication path (e.g., a HyperTransport link) to an I/O(input/output) bridge 507. I/O bridge 507, which may be, e.g., aSouthbridge chip, receives user input from one or more user inputdevices 508 (e.g., keyboard, mouse, joystick, digitizer tablets, touchpads, touch screens, still or video cameras, motion sensors, and/ormicrophones) and forwards the input to CPU 502 via memory bridge 505.

A display processor 512 is coupled to memory bridge 505 via a bus orother communication path (e.g., a PCI Express, Accelerated GraphicsPort, or HyperTransport link); in one embodiment display processor 512is a graphics subsystem that includes at least one graphics processingunit (GPU) and graphics memory. Graphics memory includes a displaymemory (e.g., a frame buffer) used for storing pixel data for each pixelof an output image. Graphics memory can be integrated in the same deviceas the GPU, connected as a separate device with the GPU, and/orimplemented within system memory 504.

Display processor 512 periodically delivers pixels to a display device510 (e.g., a screen or conventional CRT, plasma, OLED, SED or LCD basedmonitor). Additionally, display processor 512 may output pixels to filmrecorders adapted to reproduce computer generated images on photographicfilm. Display processor 512 can provide display device 510 with ananalog or digital signal.

A system disk 514 is also connected to I/O bridge 507 and may beconfigured to store stereo content and applications and data for use byCPU 502 and display processor 512. System disk 514 provides non-volatilestorage for applications and data and may include fixed or removablehard disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray,HD-DVD, or other magnetic, optical, or solid state storage devices.

A switch 516 provides connections between I/O bridge 507 and othercomponents such as a network adapter 518 and various add-in cards 520and 521. Network adapter 518 allows system 500 to communicate with othersystems via an electronic communications network, and may include wiredor wireless communication over local area networks and wide areanetworks such as the Internet.

Other components (not explicitly shown), including USB or other portconnections, film recording devices, and the like, may also be connectedto I/O bridge 507. For example, an audio processor may be used togenerate analog or digital audio output from instructions and/or dataprovided by CPU 502, system memory 504, or system disk 514.Communication paths interconnecting the various components in FIG. 5 maybe implemented using any suitable protocols, such as PCI (PeripheralComponent Interconnect), PCI Express (PCI-E), AGP (Accelerated GraphicsPort), HyperTransport, or any other bus or point-to-point communicationprotocol(s), and connections between different devices may use differentprotocols as is known in the art.

In one embodiment, display processor 512 incorporates circuitryoptimized for graphics and video processing, including, for example,video output circuitry, and constitutes a graphics processing unit(GPU). In another embodiment, display processor 512 incorporatescircuitry optimized for general purpose processing. In yet anotherembodiment, display processor 512 may be integrated with one or moreother system elements, such as the memory bridge 505, CPU 502, and I/Obridge 507 to form a system on chip (SoC). In still further embodiments,display processor 512 is omitted and software executed by CPU 502performs the functions of display processor 512.

Pixel data can be provided to display processor 512 directly from CPU502. In some embodiments of the present invention, instructions and/ordata representing a scene are provided to a renderfarm or a set ofserver computers, each similar to system 500, via network adapter 518 orsystem disk 514. The renderfarm generates one or more rendered images ofthe scene using the provided instructions and/or data. These renderedimages may be stored on computer-readable media in a digital format andoptionally returned to system 500 for display. Similarly, stereo imagepairs processed by display processor 512 may be output to other systemsfor display, stored in system disk 514, or stored on computer-readablemedia in a digital format.

Alternatively, CPU 502 provides display processor 512 with data and/orinstructions defining the desired output images, from which displayprocessor 512 generates the pixel data of one or more output images,including characterizing and/or adjusting the offset between stereoimage pairs. The data and/or instructions defining the desired outputimages can be stored in system memory 504 or graphics memory withindisplay processor 512. In an embodiment, display processor 512 includes3D rendering capabilities for generating pixel data for output imagesfrom instructions and data defining the geometry, lighting shading,texturing, motion, and/or camera parameters for a scene. Displayprocessor 512 can further include one or more programmable executionunits capable of executing shader programs, tone mapping programs, andthe like.

CPU 502, renderfarm, and/or display processor 512 can employ any surfaceor volume rendering technique known in the art to create one or morerendered images from the provided data and instructions, includingrasterization, scanline rendering REYES or micropolygon rendering, raycasting, ray tracing, image-based rendering techniques, and/orcombinations of these and any other rendering or image processingtechniques known in the art.

It will be appreciated that the system shown herein is illustrative andthat variations and modifications are possible. The connection topology,including the number and arrangement of bridges, may be modified asdesired. For instance, in some embodiments, system memory 504 isconnected to CPU 502 directly rather than through a bridge, and otherdevices communicate with system memory 504 via memory bridge 505 and CPU502. In other alternative topologies display processor 512 is connectedto I/O bridge 507 or directly to CPU 502, rather than to memory bridge505. In still other embodiments, I/O bridge 507 and memory bridge 505might be integrated into a single chip. The particular components shownherein are optional; for instance, any number of add-in cards orperipheral devices might be supported. In some embodiments, switch 516is eliminated, and network adapter 518 and add-in cards 520, 521 connectdirectly to I/O bridge 507.

Various embodiments of the present invention may be implemented as aprogram product for use with a computer system. The program(s) of theprogram product define functions of the embodiments (including themethods described herein) and can be contained on a variety ofcomputer-readable storage media. Illustrative computer-readable storagemedia include, but are not limited to: (i) non-writable storage media(e.g., read-only memory devices within a computer such as CD-ROM disksreadable by a CD-ROM drive, flash memory, ROM chips or any type ofsolid-state non-volatile semiconductor memory) on which information ispermanently stored; and (ii) writable storage media (e.g., floppy diskswithin a diskette drive or hard-disk drive or any type of solid-staterandom-access semiconductor memory) on which alterable information isstored.

The invention has been described above with reference to specificembodiments and numerous specific details are set forth to provide amore thorough understanding of the present invention. Persons skilled inthe art, however, will understand that various modifications and changesmay be made thereto without departing from the broader spirit and scopeof the invention as set forth in the appended claims. The foregoingdescription and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

1. A method for combining stereoscopic pair layers, the methodcomprising: receiving a first left eye image and a first right eye imageof a first stereo image pair layer, wherein the first left eye image isaligned with the first right eye image according to a firstpredetermined offset value used when the first stereo image pair layerwas created to produce the first stereo image pair layer that appears ata first convergence depth in a display environment; receiving a secondleft eye image and a second right eye image of a second stereo imagepair layer, wherein the second left eye image is aligned with the secondright eye image according to a second predetermined offset value toproduce the second stereo image pair layer that appears at a secondconvergence depth in the display environment that is different than thefirst convergence depth; combining the first left eye image and thesecond left eye image to produce a combined left eye image of astereoscopic image that is suitable for display; and combining the firstright eye image and the second right eye image to produce a combinedright eye image of the stereoscopic image.
 2. The method of claim 1,wherein the first convergence depth is coincident with a display surfacein the display environment.
 3. The method of claim 2, wherein the secondconvergence depth is behind the display surface in the displayenvironment.
 4. The method of claim 2, wherein the second convergencedepth is in front of the display surface in the display environment. 5.The method of claim 1, further comprising the steps of: receiving athird left eye image and a third right eye image of a third stereo imagepair layer, wherein the third left eye image is aligned with the thirdright eye image according to a third predetermined offset value usedwhen the first stereo image pair layer was created to produce the thirdstereo image pair layer that appears at a third convergence depth in thedisplay environment that is different than the first convergence depthand the second convergence depth; combining the third left eye imagewith the first left eye image and the second left eye image to producethe combined left eye image; and combining the third right eye imagewith the first right eye image and the second right eye image to producethe combined right eye image.
 6. The method of claim 1, furthercompromising the step of determining a characterized offset that is anoffset between the first left eye image and the first right eye image ofthe first stereo image pair layer in the display environment.
 7. Themethod of claim 6, further comprising the step of determining that thecharacterized offset does not equal the first predetermined offsetvalue.
 8. The method of claim 7, wherein the first predetermined offsetvalue is greater than the characterized offset.
 9. The method of claim7, wherein the first predetermined offset value is less than thecharacterized offset.
 10. The method of claim 7, wherein thecharacterized offset is determined to not equal the first predeterminedoffset value when the characterized offset is not within a threshold ofthe first predetermined offset value.
 11. The method of claim 6, whereina ratio of the first predetermined offset value to the characterizedoffset equals a ratio of a dimension of an intended display surface usedwhen the first stereo image pair layer was created to the dimension of adisplay surface in the display environment.
 12. The method of claim 11,wherein the dimension is a horizontal width.
 13. The method of claim 6,wherein the step of determining a characterized offset comprisesdisplaying a stereoscopic test pattern in the display environment. 14.The method of claim 13, wherein the step of determining a characterizedoffset comprises measuring a characteristic of the stereoscopic testpattern in the display environment.
 15. The method of claim 13, whereinthe characterized offset is encoded as part of the stereoscopic testpattern and a value of the characterized offset is indicated when thestereoscopic test pattern is displayed in the display environment. 16.The method of claim 6, wherein step of determining a characterizedoffset comprises increasing or decreasing the offset until the offsetequals the first predetermined offset value to determine an adjustmentfactor that can be applied to the offset to produce the firstpredetermined offset value during the display of the first stereo imagepair layer in the display environment.
 17. The method of claim 1,further comprising the steps of: determining that the firstpredetermined offset value has changed from a first value to a secondvalue between a first frame and a second frame; and aligning a left eyeimage and a right eye image of a first stereo image pair layer for thesecond frame according to the changed first predetermined offset valueto produce an aligned first stereo image pair layer for the second framethat appears at a third convergence depth, the third convergence depthbeing different than the first convergence depth.
 18. The method ofclaim 1, wherein the first predetermined offset value is encoded withthe first stereo image pair layer and the second predetermined offsetvalue is encoded with the second stereo image pair layer.
 19. The methodof claim 1, wherein a width of an intended display surface used when thefirst stereo image pair was created is different than a width of adisplay surface in the display environment.
 20. A non-transitorycomputer readable medium storing instructions for causing a processor tocombine stereoscopic pair layers by performing the steps of: receiving afirst left eye image and a first right eye image of a first stereo imagepair layer, wherein the first left eye image is aligned with the firstright eye image according to a first predetermined offset value usedwhen the first stereo image pair was created to produce the first stereoimage pair layer that appears at a first convergence depth in a displayenvironment; receiving a second left eye image and a second right eyeimage of a second stereo image pair layer, wherein the second left eyeimage is aligned with the second right eye image according to a secondpredetermined offset value to produce the second stereo image pair layerthat appears at a second convergence depth in the display environmentthat is different than the first convergence depth; combining the firstleft eye image and the second left eye image to produce a combined lefteye image of a stereoscopic image that is suitable for display; andcombining the first right eye image and the second right eye image toproduce a combined right eye image of the stereoscopic image.
 21. Thenon-transitory computer readable medium of claim 20, wherein a width ofan intended display surface used when the first stereo image pair wascreated is different than a width of a display surface in the displayenvironment.
 22. A stereoscopic image system, comprising: a memoryconfigured to store instructions for combining a first stereo image pairlayer and a second stereo image pair layer; and a display processorconfigured to: receive a first left eye image and a first right eyeimage of a first stereo image pair layer, wherein the first left eyeimage is aligned with the first right eye image according to a firstpredetermined offset value used when the first stereo image pair wascreated to produce the first stereo image pair layer that appears at afirst convergence depth in a display environment; receive a second lefteye image and a second right eye image of a second stereo image pairlayer, wherein the second left eye image is aligned with the secondright eye image according to a second predetermined offset value toproduce the second stereo image pair layer that appears at a secondconvergence depth in the display environment that is different than thefirst convergence depth; combine the first left eye image and the secondleft eye image to produce a combined left eye image of a stereoscopicimage that is suitable for display; and combine the first right eyeimage and the second right eye image to produce a combined right eyeimage of the stereoscopic image.
 23. The system of claim 22, wherein thefirst convergence depth is coincident with a display surface in thedisplay environment.
 24. The system of claim 23, wherein the secondconvergence depth is behind the display surface in the displayenvironment.
 25. The system of claim 23, wherein the second convergencedepth is in front of the display surface in the display environment. 26.The system of claim 22, wherein the display processor is furtherconfigured to: receive a third left eye image and a third right eyeimage of a third stereo image pair layer, wherein the third left eyeimage is aligned with the third right eye image according to a thirdpredetermined offset value used when the first stereo image pair layerwas created to produce the third stereo image pair layer that appears ata third convergence depth in the display environment that is differentthan the first convergence depth and the second convergence depth;combine the third left eye image with the first left eye image and thesecond left eye image to produce the combined left eye image; andcombine the third right eye image with the first right eye image and thesecond right eye image to produce the combined right eye image.
 27. Thesystem of claim 22, wherein the display processor is further configuredto determine a characterized offset that is an offset between the firstleft eye image and the first right eye image of the first stereo imagepair layer in the display environment.
 28. The system of claim 27,wherein a ratio of the first predetermined offset value to thecharacterized offset equals a ratio of a dimension of an intendeddisplay surface used when the first stereo image pair layer was createdto the dimension of a display surface in the display environment. 29.The system of claim 22, wherein the system further comprises a projectorthat is coupled to the display processor and configured to project thestereoscopic image onto a display surface within the displayenvironment.
 30. The system of claim 29, wherein the projector isfurther configured to display a stereoscopic test pattern in the displayenvironment.
 31. The system of claim 22, wherein the display processoris further configured to: determine that the first predetermined offsetvalue has changed from a first value to a second value between a firstframe and a second frame; and align a left eye image and a right eyeimage of a first stereo image pair layer for the second frame accordingto the changed first predetermined offset value to produce an alignedfirst stereo image pair layer for the second frame that appears at athird convergence depth, the third convergence depth being differentthan the first convergence depth.
 32. The system of claim 22, whereinthe first predetermined offset value is encoded with the first stereoimage pair layer and the second predetermined offset value is encodedwith the second stereo image pair layer.
 33. The system of claim 22,wherein a width of an intended display surface used when the firststereo image pair was created is different than a width of a displaysurface in the display environment.